A Modern Take on the Bias-Variance Tradeoff in Neural Networks

The bias-variance tradeoff tells us that as model complexity increases, bias falls and variances increases, leading to a U-shaped test error curve. However, recent empirical results with over-parameterized neural networks are marked by a striking absence of the classic U-shaped test error curve: test error keeps decreasing in wider networks. This suggests that there might not be a bias-variance tradeoff in neural networks with respect to network width, unlike was originally claimed by, e.g., Geman et al. (1992). Motivated by the shaky evidence used to support this claim in neural networks, we measure bias and variance in the modern setting. We find that both bias and variance can decrease as the number of parameters grows. To better understand this, we introduce a new decomposition of the variance to disentangle the effects of optimization and data sampling. We also provide theoretical analysis in a simplified setting that is consistent with our empirical findings

Clinical Features Present, Past & Future Prospective of Monkey Pox: A Orthopoxvirus

Some issues regarding the potential spread of monkeypox have arisen just as the international world is beginning to recover from the initial alarm that was caused by the probable spread of coronavirus disease 2019 (COVID-19). Despite the fact that parts of Africa have traditionally been more susceptible to monkeypox than other regions of the world, the majority of new cases that have been linked to the outbreak that began in 2022 have been reported in countries located in Europe and the western hemisphere. Despite the fact that a great number of organisations are working on contact-tracing activities at the moment, the origin of this outbreak is still unknown at this time. The monkeypox virus belongs to the family of viruses known as Poxviridae and the genus known as Orthopoxvirus. Following the eradication of smallpox across the globe in the 1970s, news of monkeypox caused widespread worry across the globe. Through vaccination with the smallpox virus, individuals were able to develop cross-immunity against monkeypox. After distribution of the smallpox vaccine was discontinued, the number of outbreaks of monkeypox rose. The monkeypox epidemic that occurred in the United States in 2003 was the first time that the disease gained extensive notice in the media. In spite of its name, the virus known as monkeypox is not transmitted by monkeys. Although a number of different kinds of rodents and other small mammals have been suggested as the primal hosts of the monkeypox virus, the virus\u27s true lineage is still a mystery. The virus that causes monkeypox was first identified in macaque monkeys, which is where the disease was first seen. When monkeypox does transfer from one person to another, it often does so through a person\u27s mucocutaneous lesions or through the respiratory droplets that they expel. However, this only happens very infrequently. However, supporting therapy can be given to reduce symptoms, and medications such tecovirimat may be administered in really severe cases. At this time, there is no specific treatment for patients who have infected the virus; however, supportive treatments can be given. It is debatable whether or not these treatments are successful in reducing symptoms because there are no concrete guidelines to follow in this regard

MixupE: Understanding and Improving Mixup from Directional Derivative Perspective

Mixup is a popular data augmentation technique for training deep neural networks where additional samples are generated by linearly interpolating pairs of inputs and their labels. This technique is known to improve the generalization performance in many learning paradigms and applications. In this work, we first analyze Mixup and show that it implicitly regularizes infinitely many directional derivatives of all orders. Based on this new insight, we propose an improved version of Mixup, theoretically justified to deliver better generalization performance than the vanilla Mixup. To demonstrate the effectiveness of the proposed method, we conduct experiments across various domains such as images, tabular data, speech, and graphs. Our results show that the proposed method improves Mixup across multiple datasets using a variety of architectures, for instance, exhibiting an improvement over Mixup by 0.8% in ImageNet top-1 accuracy.Comment: 16 pages, Best Student Paper Award at UAI 202

Improving Accuracy and Efficiency of Object Detection Algorithms Using Multiscale Feature Aggregation Plugins

In this paper, we study the use of plugins that perform multiscale feature aggregation for improving the accuracy of object detection algorithms. These plugins improve the input feature representation, and also remove the semantic ambiguity and background noise arising from feature fusion of low and high layers representation. Further, these plugins improve focus on the contextual information that comes from the shallow layers. We carefully choose the plugins to strike a delicate balance between accuracy and model size. These plugins are generic and can be easily merged with the baseline models, which avoids the need for retraining the model. We perform experiments using the PASCAL-VOC2007 dataset. While the baseline SSD has 22M parameters and an mAP score of 77.20, the use of the SFCM (one of the plugins we used) increases the mAP score to 78.82 and the number of parameters to 25M. © Springer Nature Switzerland AG 2020

Leveraging Synthetic Targets for Machine Translation

In this work, we provide a recipe for training machine translation models in a limited resource setting by leveraging synthetic target data generated using a large pre-trained model. We show that consistently across different benchmarks in bilingual, multilingual, and speech translation setups, training models on synthetic targets outperforms training on the actual ground-truth data. This performance gap grows bigger with increasing limits on the amount of available resources in the form of the size of the dataset and the number of parameters in the model. We also provide preliminary analysis into whether this boost in performance is linked to ease of optimization or more deterministic nature of the predictions, and whether this paradigm leads to better out-of-distribution performance across different testing domains